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1. Introduction 

Recently the dynamics of extremely diluted symmetric Q-Ising neural networks has 
been solved completely M. In spite of the extremely diluted architecture, precisely the 
symmetry causes feedback correlations from the second time step onwards, in contrast 
with an asymmetric architecture §], complicating the dynamics in a nontrivial way. 
Based upon the time evolution of the distribution of the local field a recursive scheme 
has been developed in order to calculate the evolution of the relevant order parameters of 
the network at zero temperature. Furthermore, by requiring that the local field becomes 
time-independent implying that some correlations are neglected, fixed-point equations 
are obtained and the capacity-gain parameter phase diagrams for the Q = 2, 3 models 
have been discussed shortly. 

For the case of Q — 2 it turns out that the resulting fixed-point equations from 
this dynamic approach are the same as those derived from a thermodynamic replica- 
symmetric mean-field theory treatment given in ref. Q. This gives us some insight 
concerning a possible relation between replica symmetry and the structure of the noise. 
Of course, it would be interesting to know whether this stays valid for Q > 2. Besides, a 
detailed discussion of the phase diagram for these extremely diluted symmetric models 
is interesting on its own. However, for these cases a thermodynamic approach is not yet 
available in the literature. The purpose of this work is precisely to present the results 
of such an approach. 

Concretely, we consider the extremely diluted symmetric Q-Ising neural network 
with arbitrary gain parameter. Using the replica-symmetric mean-field approximation 
we investigate both its thermodynamic and retrieval properties at zero and non-zero 
temperatures. Explicit results are presented for Q = 3, 4, and Q = oo. 

The rest of this paper is organized as follows. In section 2 the model is introduced 
from a dynamical point of view. Section 3 presents the replica-symmetric mean-field 
approximation and obtains the relevant fixed-point equations for general Q. In section 4 
these equations are studied in detail for arbitrary temperatures and Q = 3 (section 4.1), 
Q = 4 (section 4.2) and Q = oo (section 4.3). In particular, the storage capacity 
as a function of the gain parameter and capacity-temperature phase diagrams are 
obtained. The specific thermodynamic properties are discussed. The results turn out 
to be significantly different for odd and even Q. They are compared with those for 
other architectures of these Q-Ising models. Section 6 presents the concluding remarks. 
Finally, the appendix contains the specific fixed-point equations for the different values 
of Q treated in the paper. 

2. The model 

Consider a network of iV neurons which can take values <j\ in the set of equidistant 

states 



S Q = {s k = -1 + 2(k - 1)/(Q - 1), k = 1, . . . , Q}. 



(1) 
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In this network we want to store p patterns /i = 1, . . . ,p} that are supposed to 
be independent and identically distributed random variables (i.i.d.r.v.) with zero mean 
and variance A. The latter is a measure for the activity of the patterns. 
Given a configuration er = (<Ti, . . . , <7/v), the local field hi of neuron % is 

W*) = J i3 a i > ( 2 ) 

with Jij the synaptic couplings between neurons i and j. 

The network is taken to be extremely diluted but symmetric meaning that the 
couplings are chosen as follows. Let {cij = 0,1}, i,j = 1,...,N be i.i.d.r.v. with 
distribution Prjc^ = x} = (1 — c/N)8 X} o + (c/N)S X) i and satisfying c^- = Cji, Qj = 0, 
then 

^ = f c tm fOT Mi- (3) 

We remark that compared with the asymmetrically diluted model the architecture 
is still a local Cayley-tree but no longer directed. In the limit iV — > oo the probability 
that the set of connections, Tj = {j = 1, . . . , N\cij = 1}, giving information to the site 
i, is equal to a certain number k remains a Poisson distribution with mean c = £?[|Tj|]. 
Thereby it is assumed that c <C log AT. In order to get an infinite average connectivity 
allowing to store infinitely many patterns p, one also takes then the limit c —>■ oo and 
defines the capacity a by p = ac. However, although for the asymmetric architecture, at 
any given time step t all spins are uncorrelated and hence no feedback is present, for the 
symmetric architecture this is no longer the case, causing feedback from t > 2 onwards 
0, ||. Indeed, if the coupling is non-zero then also Jji is non-zero and therefore the 

state of neuron i at time t depends on its state at time t — 2,t — 4, This is not 

the case for asymmetric dilution since the probability to have a non-zero Jji given a 
non-zero J^, vanishes. 

The neurons are updated asynchronously according to the transition probability 

exp[-(3ei(s k \hi(a))] 
T,?=iGxp[~f3ei(si\hi(cr))} 

Here the inverse temperature (3 = T _1 measures the noise level, and the energy potential 
€i(s\h) is taken to be 

€i ( s \h) = -hs + bs 2 b>0 (5) 

At zero temperature, o~\ takes the value s^ leading to the minimum of the energy 
potential. This is equivalent to using an input-output relation 

a\ = gb[hi(tr)] 
Q 

9b{x) = s k[d(b(s k+1 + s k ) - x) - 9(b(s k + s fc _i) - a:)] (6) 

k=l 



Pr((7 i = 8 k \tr) = r / 17 / \\i • ( 4 ) 
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with s = — oo and sq+i = oo. For finite Q this input-output relation has a steplike 
shape and the parameter b controls the steepness of the steps. For Q = oo the input- 
output function (Q) becomes the piecewise linear function 

, s f sign(x) if \x\ > 2b 

= { i otherwise. (7) 

The slope of the linear part is given by (2b)~ l . In general, as b goes to zero the input- 
output relation reduces to that of the Ising-type network independent of Q. 

In what follows we present a detailed study of the properties of these symmetrically 
diluted networks function of T and b. 



3. Replica-symmetric mean-field theory 



The longtime behaviour of the network under consideration is governed by the 
Hamiltonian 



(8) 



In order to calculate the free energy we use the standard replica method as applied 
to dilute spin-glass models ||, [|, §, |l(| . Starting from the replicated partition function 
averaged over the connectivity and the non-condensed patterns we arrive at 



(z n ) c = n 


J dm» 


n 


J dq^yi 


n 


J dq y 


M>7 




7,7' 




7 





exp [-Nf3f] 



/ = 4E K) 2 + ^ E & + T- £ - \ «H-> Tr exp [-(3H 



2 ^ 7 /? 

M>7 " 7<7' 7 ^ 

^ = - E m 7^ a 7 - P a E Qll'^l' + E ( 6 - -TT^7 J ^7 



{a 



/•»>7 7<7' 7 V / 

where the m^, g 77 >, g 7 are the usual order parameters defined by 
1 



m 7 = ^E^( a 7,*) 7=1,..., 71 



^77' = ]v E K,*> (°Y,i) 7 ^ 7 = 1, • • • , n 



?7=^E 



a. 



7>* 



7 = l,...,n 



(9) 
(10) 

(11) 

(12) 
(13) 
(14) 



with n the number of replicas and the sum over fi running over the number of condensed 
patterns s. Assuming replica symmetry, i.e., = m M ,g 77 / = q,q y = q the free energy 
becomes 

f((3) = lim/ A 



Of. 



E ( m »Y + TT^X" + -<?X 



2 V ' 4/?" ' 2 



1 





"Dz l n {cr} Tr exp 



(3a E^^ + V®q z ~ ha 



(15) 
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with b = b — £x the effective gain parameter, \ = ft{q ~ q) the susceptibility and 
T>z = dz (27r)~ 1//2 exp(— z 2 /2) the Gaussian measure. We remark that the effective gain 
parameter b can be negative, implying that the input-output function reduces to that 
of 2-Ising-type neurons, i.e., g~ b {h) = sign(/i). Furthermore, we note that the free energy 
(|T5|) can be obtained as the formal expansion to second order in x of the free energy for 
the fully connected model derived in [TT|. 

The phase structure of the network is determined by the solution of the fixed-point 
equations for the order parameters 

m„=j(^Dzenz)))) (16) 

q = ((/ ' Dz{a{z)) 2 )) (17) 

X =-}=((fDzz <„«>)) (18) 



taq 

which extremize —f3f{(3). Here 

/ ,xv = Tr fT aexp[/3o-(E M m M ^+ y/aqz-ba)} 
Ti s exp[(3s(X,i,m^ + ^/aqz -is)] 
In the following section we discuss these equations for Q = 3, 4 and Q = oo models. 

4. Thermodynamic and retrieval properties 

4.1. Q = 3 

In the rest of this work we are mainly interested in the Mattis retrieval state so that we 
consider one condensed pattern, say the first one. Then we can write m M = m<5 M i and, 
furthermore, we leave out the index 1 in the sequel. 

Let us consider a three-state network with uniformly distributed patterns taking 
the values 0, ±1 such that A = 2/3. For the Mattis retrieval state the average (|T9|) is 
given by 



(o-(z)) = - ^mmj + ^qz)] {2Q) 

2 exp(/36) + cosh[/3(m£ + y/aqz)] 



which reads at zero temperature 

(a(z))=g- b (mt + y/aqz). (21) 

The fixed-point equations (|i"6D-fli"8D can then easily be written down and the 
integration can be performed explicitly at zero temperature. For completeness we 
present them in the Appendix. 

First we look at the network at zero temperature. For b < these fixed-point 
equations can be further reduced by taking the limit b — > and introducing the variable 
x = mj ^2aq. One arrives at 

erf fx) . , 

2a = — — 22 

x 
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with, in view of the definition of b = b — ~x 



b<b = ^-^ [2exp(-x 2 ) + lj . (23) 

This learns us that the retrieval state vanishes continuously at a c = 2/n for b < bo = 
1/tt. 

The phase boundary of the retrieval state for b > can be obtained by numerically 
solving the fixed-point equations ( |A.5| )- (rA~7| ). The results are shown in the capacity-gain 
parameter phase diagram presented in figure [1] as the full and the long-dashed curve. 
The full curve denotes a continuous appearance of the retrieval state, the long-dashed 
curve a discontinuous one. Furthermore, the dotted curve separates the 2-Ising-like 
region (which is to the left) where b < 0. 

Finally, the short-dashed line in this diagram indicates the boundary above which 
the spin-glass solution exists. The spin glass is characterized by taking m = and 
q ytz in ( [A.5| )-( [A~TD . The resulting fixed-point equations can be combined into a single 
equation in the variable y = bj \J1aq 

h yJT^j) + ,. eXp(V) .., . (24) 



' 2^/Ml-erf(i,))]' 

Since the right-hand side of the above equation is bounded, a spin-glass solution is 
possible when a > cb 2 with c ~ 1.0134. It vanishes discontinuously at the boundary. 

In order to find the thermodynamic transition line we have to determine which 
state - retrieval, spin- glass or paramagnetic - leads to the lowest free energy. At zero 
temperature the free energy is given by 
A a 

f = -—m 2 - -qx + bq (25) 

as long as x is finite. This is certainly the case when q is nonzero. Since also q = 
for the paramagnetic state, however, it is necessary to check whether x is finite. The 
paramagnetic state is determined by the equation 

b = b °t . (26) 

exp(/%) + 2 

This equation has three solutions. In the limit T —>■ 0, only the solution that converges to 
b with x ~ 2/3exp(— /3b) — > is stable against longitudinal fluctuations as is easily seen 
by expanding the free energy for small q. The free energy of this stable paramagnetic 
state is always zero. The retrieval state becomes the global minimum of the free energy 
in the region bounded by the thick full curve. So, for a = the retrieval state is the 
minimum for b G [0, 1/2]. The latter is true independent from the architecture. To the 
right of the thick full curve the paramagnetic state is the global minimum. We remark 
that this paramagnetic state at T = is in fact simply a frozen state with all the spins 
taking the value zero (and not a phase where all the spins take all possible values with 
the same probability). 



Thermodynamic properties of extremely diluted symmetric Q-Ising neural networks 7 



Finally, the stability of the replica symmetric retrieval solution against replica- 



symmetry breaking is determined by the replicon eigenvalue [12 




Xr = 1- P 2 a(( I Vz [(a\z)) - {a{z)f) 2 )) ^ (27) 

(28) 

where (o~(z)) is given by eq. (2D). In the limit T — > 0, (o~(z)) becomes a step function 
so that the argument of the integration over z is proportional to the square of the delta 
function. Therefore the replicon eigenvalue for the retrieval solution at T = is always 
negative and, hence, replica symmetry is broken. 

For the paramagnetic solution that we have discussed before we find that the 
replicon eigenvalue \pm = 1 so that this state is also stable against transverse 
fluctuations. 

At this point a couple of remarks are in order. First, we note that the fixed-point 
equations and, hence, also the a — b phase diagram are precisely the same as those 
obtained from an exact dynamical approach (see U eqs. (23)-(25) and fig. 2) after 
requiring that the evolution of the distribution of the local field becomes stationary at 
a certain time t. This requirement has the consequence that most of the discrete noise 
caused by the feedback in the retrieval dynamics is neglected. It would mean that for 
these models a replica-symmetric mean-field theory treatment corresponds to a specific 
simplification of the structure of the noise - only the Gaussian part plus the discrete 
noise coming from the last time step - induced by the non-condensed patterns. This 
is also related to the so called self-consistent signal-to-noise-ratio analysis introduced 
13. However, more work is needed, also on other models, to put such a type of 



in 



conjecture on a firm basis. Second, it is interesting to compare the phase diagram figure 
with the one for the extremely diluted asymmetric architecture The essential 
difference is that the 2-Ising-like region is much more extended here. The rest of the 
diagram has, in fact, a similar shape but it is tilted because of this Ising-like region 
towards greater b- values (compare || fig. 1). 

Next we consider the Q = 3 network at non-zero temperatures. The results are 
presented in figures |2| and |3|. From T = 0.37 onwards the retrieval state appears 
continuously at the whole boundary. Furthermore, the retrieval state is the global 
minimum of the free energy in a growing region covering the whole retrieval region of 
the phase diagram as the temperature increases to T = 0.5. For T = 0.6 the lower 
branch of the retrieval boundary ends in zero such that for higher temperatures we 
apparently have no retrieval for small values of a. For T = 0.7 and 6 = 0, e.g., a > 0.2. 
Finally, the replica symmetry solution is no longer unstable in the whole retrieval region. 
Calculating the replicon eigenvalue ( p8| ) leading to the de Almeida-Thouless (AT) line 
(dashed-dotted curve in figures |2| and ^) we find a growing region of stability in the 
phase diagram below this AT-line. 

In order to give a more complete idea of these results we present the capacity- 
temperature phase diagrams in figure |||. We immediately notice that for higher 
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temperatures we get a higher critical capacity, in other words the upper branch of the 
retrieval boundary shows some strong re-entrant behaviour. This is not surprising since 
this branch lies completely in the region where replica symmetry is broken, in contrast 
with the fully-connected (symmetric) model ||11|| . It is common knowledge, at least for 
Q = 2 j|, [|, that re-entrance is due to the use of the replica-symmetric approximation. 
It is also conjectured there that a full replica symmetry breaking solution might be 
a horizontal line above this upper branch of the retrieval boundary starting from the 
high temperature point. Validating this conjecture by numerical simulations or doing a 
one step replica symmetry breaking calculation in order to see whether such results lie 
indeed closer to this horizontal line are beyond the scope of the present work. 



12. Q = A 

The thermodynamic properties of a network consisting of neurons that are able to take 
on the zero-state (<Tj = 0) are significantly different from those of a network in which 
this state is forbidden for the neurons. Indeed, for the even Q models a paramagnetic 
phase at zero temperature has to be absent precisely because the spins can not take the 
value zero and, hence, q cannot be zero physically. For high temperatures, of course, 
q can be zero and such phase does exist. This is similar to the asymmetric diluted 
architecture fU and the fully connected architecture |]lT| . As a representative example 
we consider the Q = 4 model in this section. 

According to ([!]) the neurons, as well as the patterns, can take on the values 
— 1,-1/3, +1/3, +1. We consider uniformly distributed patterns so that A = 5/9. 

At zero temperature, the fixed-point equations for a retrieval state with b > 
are given in the appendix. For b < 0, these equations can be further reduced by the 
introduction of the variable x = m/y/2aq to 



9 



erf(x) H — erf(-a;) 
3 3 



with, in view of the definition of b 



b<b 



exp(— x ) + exp( 



1 

9' 



-x 



(29) 



(30) 



Exactly as in the Q = 3 case the retrieval state vanishes continuously at a c = 2 /it 
for b < bo = In contrast with Q — 3, however, the fixed-point equations have 

retrieval solutions for all values of b. When b — > 00, they can also be reduced to a 
single equation similar to eq. (P9"|). This means that the retrieval state again vanishes 
continuously at a c = 2/tt. For the other values of b the retrieval phase boundary can 
be obtained numerically from ( |A.8] )-( |A.10| ). The result is indicated in figure |3] by a full 
line expressing the fact that the retrieval state appears continuously everywhere. 

The triangular region in figure f| bordered by the thin long-dashed curve indicates 
the existence of two retrieval states with different free energies. Both stable retrieval 
states have the same free energies at the thick long-dashed curves. In order to get an 
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idea about the relevant b and m-values for this case, we note that for a = 







b 


< 


1/4 


m = 


6/5 




1/4 


< 


b 


< 


3/10 


m = 


6/5 


and 1 


3/10 


< 


b 


< 


3/4 


m = 


1 


and 2/5 


3/4 


< 


b 






m = 


2/5 





(31) 



Again, these results are valid independent of the architecture (compare [§, [Hp. 

Concerning the spin-glass states we have to solve the equations ( |A.8| )-( |A.10|) for 
m = 0. It is straightforward to check that a spin-glass phase exists in the whole region 
of the phase diagram. By looking at the free energy expression (p5|) we find that it is 
always energetically unfavourable versus the retrieval state (only versus the one with 
the lower free energy in case there are two). Furthermore, the retrieval state is unstable 
against replica symmetry breaking. 

Comparing the phase diagram of the Q = 4 model with that of the asymmetrically 
diluted model we see that similar to the Q = 3 case there is an extended 2-Ising- 
like region here. Again the overall shape is similar apart from the fact that it is tilted 
towards greater b- values (compare fig. 4). 



4.3. Q 



Finally we turn to the case Q = oo. Considering again uniformly distributed patterns 
between —1 and 1 (and hence A = 1/3) the fixed-point equations are still given by 
(0)-([g) with (0) replaced by 

/_! da a exp[f3a(J2u + ^far z — ba)] 



j\ ds exp[/?s(E M rn^ + 



ar z 



bs)\ 



(32) 



For the retrieval state at zero temperature this leads to the explicit fixed-point 
equations presented in ( |A.11| )- (|A.13| ) of the Appendix. These equations are written 
down for b > 0. Again, for b < the equations can be further reduced by introducing 
the variable x = raj ^/laq: 



erf(x) 



x 



1 



1 

2x 2 J 



+ 



7T X 



2 eX Pl~ 



X 



together with, in view of the definition of b, the following condition 

fa erf(x) 



b < fen 



x 



(33) 



(34) 



Exactly as in the Q = 3 and 4 model, these equations tell us that the retrieval state 
vanishes continuously at a c = 2/-7T for 6 < fe = 1/tc. In fact, one can analytically show 
that the critical boundary of this 2-Ising like region is independent of Q. Indeed, in this 
region the general fixed-point equations (|T6|)-(]T9|) at T = together with the condition 
b < lead to 

1 



Ax 



«£erf(xO» 



~ V 2tt 



exp(-x f 



x E [0, oo] . 



(35) 
(36) 
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Noting that the r.h.s. of eq. (B5|) is monotonically decreasing for x > 0, one immediately 
concludes that the phase boundary for this state is given by x —>■ so that a c = 2/n 
and 6 = regardless of Q. 

For b > the region where the retrieval solution exists is found numerically by 
solving the fixed-point equations ( |A.ll| )-( |A.13j ). The result is shown in figure [5] as the 
full line. The solution appears continuously. Compared with all other architectures 
treated in the literature -asymmetrically diluted, layered and fully connected- where 
the storage capacity is zero for b > 1/2, this is only the case here at finite loading 
a = 0. Because of the tilting of the phase diagram in comparison with the one for the 
asymmetrically diluted model, as seen already for Q = 3,4, this is no longer the case 
for non-finite loading. The retrieval state is the global minimum of the free energy in 
the whole retrieval region. 

For a discussion of the spin-glass phase one can follow a similar argumentation as 
for the Q = 3 model starting from the fixed-point equation analogous to eq. ([24]). One 
finds that the region of existence of the solution is bounded below by a > b 2 , i.e. the 
dashed curve in figure |5|. At the boundary x = 1/6 and the spin-glass state appears 
continuously. 

In order to determine stability of the retrieval solution against replica symmetry 
breaking we calculate the replicon eigenvalue using eq. fl2"8|). The result reads 



A fl = l --£-((/ Vz)) =1-^ (37) 

46 2 \\J\m^+^aqz\<2b / / 26 

where we have used the fixed-point equation (|A.13 ) for x- 111 contrast with the Q = 3 
model where the replicon eigenvalue is always negative and hence breaking occurs in the 
whole retrieval region we find that there is only partial breaking here. The AT boundary 
above which the replicon eigenvalue is positive and, hence, breaking occurs, is shown as 
the dashed-dotted curve in figure [5[ 

Finally, we study the Q = oo model at non-zero temperatures. The retrieval phase 
is obtained by numerically solving the fixed-point equations (|I6|)-(|T9|). The result is 
presented as the full line in figure ||. The retrieval state appears continuously and the 
lower branch of the retrieval boundary ends in zero for 6 = 0.5. For smaller 6 it crosses 
the T-axis, e.g, at T = 1/3 for 6 = 0. 

Next, we find the boundary of the spin-glass phase by expanding the relevant 
equations (|I6|) -(|19"1) for m = with respect to q. We obtain 

"sg = Xo 2 ( 38 ) 

with 

f\do-o- 2 exp(-(3b~ o- 2 ) ~ 1 

Xo = P— . - „. , b = b-— . (39) 

j_ 1 daexp{—fjb a^j ^Xo 

The result is the short-dashed line in figure ^ above which spin-glass solutions exist. 

Concerning the paramagnetic states the same reasoning can be followed as for 
the Q = 3 model after eq. (|26|) leading to one stable solution, x ~ V&> with respect 
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to longitudinal and transverse fluctuations. However, this solution becomes unstable 
against transverse fluctuations in the spin-glass region. 

Comparing the relevant free energies we find that the retrieval states are the global 
minima in the whole retrieval region. Finally, eq. (|28| ) tells us that above the dashed- 
dotted AT-line the replica symmetric retrieval solution is unstable. Again the whole 
upper-branch of the retrieval boundary lies in this region. Except for the relevant 
temperature scale these results show some qualitative similarities with the corresponding 
Q = 3 results. 

5. Concluding remarks 

We have considered both the thermodynamic properties and retrieval properties of 
symmetrically diluted QTsing networks. Fixed-point equations have been derived for 
general temperature and arbitrary Q in the replica-symmetric mean-field approximation. 
For Q = 3, 4, and Q = oo capacity- gain parameter diagrams and capacity-temperature 
phase diagrams have been discussed in detail. 

Concerning the capacity-gain parameter diagrams we find that the results are 
essentially different for odd and even Q. Furthermore, there are interesting similarities 
with the asymmetric extremely diluted versions of the models. In fact, we find that 
the phase diagram here is tilted towards higher 6-values because of the presence of an 
extended 2-Ising-like region. The critical boundary of this region is independent of Q. 
Finally, the phase diagram for Q = 3 is precisely the same as the one obtained using an 
exact dynamical approach after requiring that the evolution of the distribution of the 
local field becomes stationary. 

Looking at the a— T phase diagrams we immediately notice the overall qualitatively 
similar behaviour of the Q = 3 and Q = oo model. Furthermore the whole upper-branch 
of the retrieval boundary lies in the replica symmetric unstable region, in contrast with 
the fully-connected model. 
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Appendix 

In this appendix we write down explicitly the fixed-point equations ([U])-([n|) for Q = 3, 
Q = 4 and Q = oo. For a three-state network with patterns taking the values 0, ±1 
with equal probability ,one obtains for a retrieval state 
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Q 

X 



Dz 



~V*(m + ^qz,b) + ~V$ 

2 , r . \ 

Dzz 



aq z, b) 



where 



V p (x,y) = T 



-Vp(m + y/aqz,b) + ~Vp(y/aqz,b) 



2 exp (/ft/) + cosh(/ftc) 



(A.2) 
(A.3) 

(A.4) 



For zero temperature the Gaussian variable z can be integrated out explicitly. For b > 
one arrives at 
1 



rn 



erf 



/ m + b\ 



+ erf^ 



(m — b\ 
^/2a~q) 



i-i 

3 



erf 



f m + b\ 



fm — b\ 



\ ^/2a~q) 6rf V V2a~q) + ^ V ^2a~q 



X 



9naq 



cxp 



(m + bf 
2aq 



+ exp 



(m — by 
2aq 



exp 



2aq) 



(A.5) 
(A.6) 
(A.7) 



For a four-state model in which the patterns can take the value ±1/3, ±1 with equal 
probability the fixed-point equations (^6|)-(p!9|) for a retrieval state at zero temperature 
read 
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18ag 



(A.10) 



Again, the above expressions are valid only for b > 0. 

Finally, for Q = oo and uniformly distributed patterns (A = 1/3) a retrieval state 
is given by the solution of the fixed-point equations (|16|)-(|T8"|) and (|32|). For the retrieval 
state at zero temperature the neuron expectation value (cr(z)) = gi(m£ + ^/aqz), with 
the effective input-output function g given by (|7[). This allows us to perform explicitly 
the Gaussian average in the fixed-point equations resulting in 
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(A.12) 
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for positive b. We remark that it is also straightforward to perform the integration 
associated with the random patterns but there is no need to write down the resulting 
expressions. 
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Figure 1. Q = 3 a — b phase diagram for uniform patterns at T = 0. The (thin) 
full and long-dashed lines denote the boundary of the retrieval region corresponding 
to respectively a continuous and discontinuous appearance of the solution. The dotted 
line separates the 2-Ising-like retrieval region. The short-dashed line indicates the 
discontinuous appearance of the spin-glass state. The thick full line represents the 
thermodynamic transition for the retrieval state. 
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Figure 2. Q ~ 3 a — b phase diagram for uniform patterns at (a): T = 0.1 and (b): 
T = 0.5. The meaning of the lines is as in Fig. 1. The dashed-dotted curve is the de 
Almcida-Thouless line above which replica symmetry is unstable. 
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Figure 4. Q = 4 a — b phase diagrams for uniform patterns at T = 0. The meaning 
of the lines is as in Fig. 1. The triangular region is explained in the text. 
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Figure 6. Q = oo a — T phase diagram for uniformly distributed patterns and (a): 
b = 0.5 and (6): b = 0.7. The short-dashed line indicates the continuous appearance of 
the spin-glass state. The meaning of the other lines is as in Figs. 1 and 2. 



